Constructive Proposals for Dealing with Attrition: An Empirical Example∗

Survey nonresponse and attrition undermine the validity of many and possibly most econometric estimates. In this paper, we discuss simple methods for assessing the nature of the damage caused by such problems. A fundamental component of the methods we discuss is partial randomization of non–response. We evaluate the performance of simple grouped data estimators for sample selection (a là Gronau and Heckman) as well as conservative bounds (a là Manski and Lee) when nonresponse is partially randomized. We develop a graphical interpretation of the conventional sample selection estimator and use it to develop tests of overidentification. To make these ideas concrete we apply the methodology to a carefully conducted randomized controlled trial, the Moving to Opportunity (MTO) demonstration project. Two features of the MTO study are useful for our purposes. First, for a subset of outcomes the MTO study team collected administrative data for all persons, allowing for estimation of the bias associated with standard sample selection procedures. Second, thirty percent of subjects were randomly assigned to intensive follow–up. In this application the extent of the bias caused by non-response is small. However, our evidence suggests both the importance of addressing nonresponse bias and the real challenges involved, even in this ideal situation. ∗Thanks to Jane Garrison for excellent research assistance and for help and ideas from Jeff Kling, without whom the empirical example would not have been possible.